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Background / Context: 

Description of prior research and its intellectual context. 

Though it is estimated that about half of the world’s population is bilingual, the estimate for the 
United States is well below 20% (Grosjean, 2010). Amid growing recognition of the need for 
second language skills to facilitate international commerce and national security and to enhance 
learning opportunities for non-native speakers of English, many U.S. public, charter, and private 
schools have developed dual-language immersion (DEI) programs. The goal of these programs is 
to help the growing number of language minority students learn English and achieve 
academically, while giving language majority students the opportunity to develop proficiency in 
another language. 

Study of another language has been viewed by some as an important but not entirely 
essential ingredient of a world-class education system. However, others point out that dual- 
language education can be a powerful intervention for closing the achievement gap for English 
language learners (EEEs) and that it may enhance outcomes for both EEEs and English native 
speakers (Collier & Thomas, 2004; Thomas & Collier, 2003; Eindholm-Eeary & Block, 2010). 

At the same time, substantial evidence attests to the cognitive benefits of bilingualism (Bialystok 
et ah, 2008; Bialystok & Craik, 2010) and supports the relationship between early learning of 
another language and success in other academic subjects (Genesee, 2012). Though numerous 
studies have established a positive relationship between dual-language education and student 
achievement, as cited above, important questions about the relationship between the two remain. 
Prior studies, which are largely descriptive, leave open the question of selection bias because we 
do not know whether the students who gain access to dual language programs have other 
characteristics that might contribute to the observed outcome of higher academic achievement. 

Our study attempts to address this limitation using random assignment to immersion 
programs. The proposed paper presents data from the second and third years of a three-year 
study that capitalizes on random-assignment lottery data from the Portland Public Schools in 
Portland, Oregon. This urban district serves about 47,000 students and is currently the second- 
largest school district in the Pacific Northwest. 

Purpose / Objective / Research Question / Focus of Study: 

Description of the focus of the research. 

This paper presents evidence from the second and third years of a three-year, lottery-based 
study of dual-language immersion in Portland Public Schools in Oregon. It updates initial student 
achievement findings with two additional years of outcome data and also augments these 
findings with evidence about program costs and administration. The research questions 
addressed in the paper are as follows: 

(1) What is the causal effect of dual-language immersion education on student achievement in 
mathematics and reading in grades 3 through 8, on student attendance in grades K through 8, 
and on initial English language learners’ exit from ELL status in grades 1 through 8? 

(2) Does this effect differ for English language learners versus native speakers of English or by 
instructional model (90 versus 50 percent of time in the partner language)? 

(3) What does it cost to implement dual-language immersion programs relative to non- 
immersion programs? What are the components of these costs, and do they vary by 
instructional model? 
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Setting: 

Description of the research location. 

The Study takes place in the Portland Public School district (PPS) in Portland, Oregon, the 
largest school district in the Pacific Northwest. PPS serves approximately 47,000 students in 
grades pre-kindergarten through 12. The district includes 85 regular-education schools, as well as 
alternative schools, charter schools, and education services provided at other locations for 
students with special needs. The study focuses in particular on the district’s 19 dual-language 
immersion schools, which include 14 schools with Spanish programs, 3 with Japanese programs, 
3 with Russian programs, and 2 with Mandarin programs. (Two schools are home to more than 
one immersion language.) 

Population / Participants / Subjects: 

Description of the participants in the study: who, how many, key features, or characteristics. 

The Study examines seven cohorts of PPS students who entered kindergarten in the academic 
years 2004-05 through 2010-11. The study includes 3,468 lottery applicants, 1,973 of whom 
were randomly assigned to a language immersion program (the treatment condition), and 1,495 
of whom were assigned to the control condition, defined as business as usual. Economically 
disadvantaged students make up about 29% of the DEI program population, and minorities about 
30%; these groups constitute 45% and 45%, respectively, of the district as a whole. English 
language learners make up about 14% of DEI participants and 14% of the district as a whole. 
Students in the analytic dataset are in grades kindergarten through six. 

Intervention / Program / Practice: 

Description of the intervention, program, or practice, including details of administration and duration. 

Dual-language immersion programs are designed to provide both majority and minority 
language speakers with an academically challenging learning environment. Students enter these 
programs as early as kindergarten and continue through high school, studying the same academic 
content as their peers, but in two languages. Such programs aim to help students become 
bilingual, bi-literate, and academically proficient, with strong cross-cultural communication 
skills. 

In Portland, the Russian program and all but one of the Spanish programs follow a two- 
way model in which up to half of the students speak the partner language — Russian or Spanish — 
as their first language, and the remaining students speak English as their first language. Students 
in these programs receive up to 90 percent of their instruction in the partner language. The other 
PPS language programs (Japanese, Mandarin, and one Spanish program), offer a one-way 
immersion model. They serve mostly native speakers of English and deliver about half of 
instruction in the partner language and half in English. 

Research Design: 

Description of the research design. 

In an effort to obtain an unbiased estimate of the effect of immersion on student 
outcomes, the research design capitalizes on the district’s randomized lottery mechanism for 
assigning DEI program slots. By comparing the outcomes of students who won the kindergarten 
DEI lottery to students in the same lottery stratum who lost the lottery and hence were not 
admitted to a DEI program in kindergarten, we can estimate the causal effect of DEI programs 
on student outcomes. 
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Data Collection and Analysis: 

Description of the methods for collecting and analyzing data. 

We estimate the effeet of immersion on student outeomes through several model 
speeifieations. Our naive speeifieation is an ordinary least squares regression analysis that 
ineludes all preK and kindergarten students in the seven targeted cohorts, regardless of whether 
they applied to an immersion lottery. However, our causal dentification strategy capitalizes on 
the aforementioned randomization, in an effort to ensure that within any given lottery subgroup 
or stratum, the unobserved characteristics of winners and losers will be identical in expectation. 
We estimate the effects of assignment to DLI using a model that accounts for this within-block 
randomization using fixed effects for each subgroup lottery in each immersion program and each 
cohort. The model is specified as follows: 

Yij = “i + + y±Lj + Etij , G=K, 1, 2, 4, 5, 6, 7 (1) 

where the dependent variable, y?-, represents the outcome of interest at grade G for student i in 
lottery j. In other words, we are tracking the outcomes of student i longitudinally across grades 
kindergarten through 7 (noting that attendance is observed all years, ELL status as an outcome is 
observed in grades 1 through 7, and standardized test scores are observed in grades 3 through 7). 
Each lottery in j refers to a particular school, cohort, and randomization subgroup (e.g., within or 
out of neighborhood, native speaker of partner language, etc.). Treatment variable Zij is a 
dichotomous indicator of random assignment to the treatment in the lottery for student i in lottery 
j; Ly is a vector of dichotomous school x randomization subgroup x cohort lottery indicators, 
and y? is a corresponding vector of lottery fixed effects; X,- is a vector of student demographic 
characteristics, and Sj is a corresponding parameter vector; «/is an intercept term, and £';,) is a 
normally distributed, mean-zero error term with standard deviation ai. The parameter of interest 
is Pi, which represents the precision- weighted causal effect of Zij on the dependent variable, y?-. 

Because attrition from the sample is observed to be higher among those not offered an 
immersion slot (see Ligure 1), we also examine sensitivity of the intent-to-treat analysis to the 
inclusion of propensity score weights estimated separately for each grade level. To obtain an 
estimate of the average treatment effect (ATE), we construct a weighting variable for the full 
analytic sample, where the individual-level weight, w, , is calculated as follows (Austin, 


2011 ): 


W: 


i.ATE 



( 2 ) 


In equation 2, is the observed treatment status (I or 0) for student i. Variable z,^ is the 
student’s estimated propensity score, which is the estimated probability of enrolling in an 
immersion program in each grade beyond kindergarten, as a function of baseline characteristics 
including race/ethnicity, gender, free/reduced-price meal eligibility, English language learner 
status in kindergarten, special education status in kindergarten, and lottery stratum. 

Instrumental Variables Analysis 

Insofar as lottery winners and those not placed fail to comply with their assigned status, 
the intent- to- treat estimate may understate the true effect of DLI participation on students’ 
performance. To estimate the effect of DLI enrollment in kindergarten on those who complied 
with their initial lottery-assignment status (also known as the Local Average Treatment Effect, or 
LATE), we use lottery assignment status (as a function of the student’s lottery number and 
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number of slots in his/her stratum) as an instrument for actual DLI enrollment in kindergarten, 
specifying a two-stage least squares regression model. 

Cost Analysis 

To understand costs of the immersion program, we conducted interviews with 9 central 
office personnel and 14 of 19 immersion school principals, inquiring in detail about the 
allocation of staff members’ time to particular tasks and the proportion of that time allocated to 
immersion-specific tasks during the 2012-13 academic year. We also asked about teacher 
workload and expectations, parent volunteerism, fieldtrips, classroom technology, and external 
funding sources, including how each of these inputs differed between immersion and non- 
immersion programs in the schools. We then calculated immersion effort ratios by dividing the 
fraction of immersion- specific time for each task and/or school by the fraction of immersion 
students in the school. These relative input ratios are then attached to dollar estimates based on 
the interviews and on publicly available budget data from the district. 

Findings / Results: 

Description of the main findings with specific details. 

Simply comparing immersion and non-immersion students in the district and adjusting 
for baseline demographic characteristics, we find that immersion students substantially 
outperform non-immersion students on all outcomes. However, these estimates may be driven in 
part by the motivation levels and characteristics of families who choose immersion. For this 
reason, we turn to data from the pre-K and kindergarten immersion lotteries, in which students 
are randomized to immersion or non-immersion conditions. 

In some model specifications, we find positive and significant or marginally significant 
effects of immersion on reading scores in grades 4-5 and non-significant effects in other years. 
However, immersion effects in mathematics, attendance, and ELL status appear not statistically 
distinguishable from 0. Using propensity scores to statistically adjust for differences in district 
enrollment rates between lottery winners and those not placed in immersion does not markedly 
change the estimates. 

In terms of costs, the proportion of time that principals reported devoting specifically to 
their DLI programs was almost exactly proportional to the proportion of DLI students in their 
schools. The share of effort devoted to immersion programs was also relatively similar across 
schools, regardless of school level (elementary, middle, high) and instructional models. 

Conclusions: 

Description of conclusions, recommendations, and limitations based on findings. 

Lindings to date suggest that immersion may improve student achievement in reading (in 
English) without diminishing other performance and while still promoting bilingualism, but 
estimates are somewhat sensitive to model specification. Moreover, it appears that the costs of 
these programs in terms of principals’ time are not greatly out of sync with traditional 
instruction. The paper will briefly touch on logistical issues that other districts wishing to 
replicate the model might also consider. 
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Appendices 

Not included in page count. 
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